The performance of business systems in the global economy is an area of considerable interest as businesses become more disperse and applications become more complex. Decisions must be made rapidly and data systems must remain reliable and available. Reliability and performance can be a considerable issue in the face of rapid system or application scaling such as would be experienced in a merger of two large corporations or in an onset of an IT outsourcing contract.
A goal of modern IT performance engineers is to optimize business applications on quite large and complex systems with perhaps many thousands of nodes that are often widely geographically dispersed. In order to meet this goal, a performance engineer might design a test environment with actual equipment running actual business applications to be tested but on a much reduced scale from a “production” environment. The performance within the test environment is carefully measured and scaled and the performance engineer would then like to take that data and project how the business application will perform in the more complex production or projected environment. In other situations, a system may be overly stressed, with such low business application performance that the situation is detrimental to the function of the corporation. To relieve the situation, the performance engineer may be asked to troubleshoot the problem quickly. To accommodate the performance engineer a tool for quickly organizing appropriate and existing test data into a form that will answer key system questions is essential. Furthermore, rapidly visualizing the answer to the key system question in a form that optimizes the performance engineer's ability to draw conclusions and make decisions has considerable value in the art of the field.
FIG. 1 shows an example of a test network to investigate application and network performance. This example includes a network of servers, workstations, business applications, data storage devices, test devices and IP network connections between them shown as LAN 115 and Internet 105. The network of servers is comprised of application server 125 connected to LAN 115 which runs business, engineering or research applications, database server 120 connected to LAN 115 and which is a local database that organizes information of interest to the business, storage server 135 connected to LAN 115 and which holds data storage 138 that feeds the servers and to which data is backed up from the servers, remote database server 190 connected to LAN 115 via the Internet 105 and remote LAN 117 and which is geographically remote from database server 120 and serves a similar function to the local server but may house different pieces of information from different business units, and remote storage server 150 connected to LAN 115 via the Internet 105 and remote LAN 116 and which is used to keep a synchronous or asynchronous copy of the local data storage 130 to remote storage 155. A workstation 130 is shown which runs a first application client 101 and a second application client 102; workstation 130 is also connected to LAN 115. Interspersed between the LAN 115 and the various servers are network sniffer devices 140, 145, 150 and 160. There is a network sniffer device 170 between LAN 115 and the Internet 105. Network sniffer devices 175 and 185, are respectively connected between the remote data storage 155 and remote storage server 150 and between the local data storage 138 and the storage server 135. There is also a network sniffer device 180 between the Internet 105 and remote database server 190. The network sniffers function to examine data packets as they traverse the network looking for a match and logging a timestamp for each match. They will also count the number of packets that match in a given time frame and perform other such functions related to network packet timing.
There are three interesting classes of test to run on this network. The first class of test, test1, captures a network trace of an instance of a business application to establish the flow of the business process through the network. For example, application client 101 may launch a web application from workstation 130 that will require various unknown network resources. Test1 will ultimately trace the paths that the application will take through the network to find the resources. Reports from test1 will typically list the various network resources and response times.
The second class of test, test2, captures resource usage of various components of the network. For example, application client 102 utilizes workstation 130, application server 125, database server 120 and storage server 135 and remote storage server 150 to create and store a set of business transactions. Test2 will correlate the usage data on the various devices in the network to the business application run to prepare a set of resource usage reports. For example, CPU utilization on Workstation 130 would be included in that report. Fairly complex reports can be created by test2 where the business function is loaded repeatedly to examine network and resource utilization under scaling.
The third class of test, test3 captures resource usage and other correlated information from various components of the network when multiple business applications are running. For example, 3 instances of the application client 101 and 5 instances of application client 102 are run at the same time. Even more complex reports are generated by test3 tests that look at resource usage and scaling in a mixed environment.
Measured data from tests like those described can be utilized in simulation and modeling programs to predict network or system performance in different environments than the one on which the measurements were made. The performance engineer with these simulation and modeling programs can generate vast amounts of data about his network or system—modeled data that can be used to rapidly solve performance problems given the right tools to organize the data.
Several recurring questions routinely arise in analysis of system performance data and in predictive scenarios. For example, questions that could be asked in such a test environment, such as “What are the bottlenecks?”, “Are the performance objectives being met?”, or, “Will the performance objectives be met when the number of clients on the network scales to 10,000?”. Typically the performance engineer will have to manipulate a large amount of data organized in spreadsheets and text files to arrive at the answers to these and other questions. Therefore, a need exists to overcome the inefficiencies in defining the queries and performing manual manipulation of performance data to arrive at answers to routine system performance questions.
A motivation of the present invention is to present the performance engineer with a class of questions and a novel apparatus to automatically organize measured data and modeled data into forms that answer system questions clearly and concisely into a visual form using charts, graphs, and tables saving much time and effort. Additionally the present invention provides the performance engineer with flexible means of manipulating complex reports so that valuable classes of reports may be saved as projects and templates to be recreated later. The ability to conveniently save templates combined with other novel mechanisms of the present invention allows the performance engineer the capability to create new questions or categories of reports that can be optimally tailored to the network under consideration.